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Evaluation of SLS Intervention Courses 



Summary 



Introduction 

and 

Method 



In May of 1998, District Student Services requested an evaluation of the new 
SLS intervention courses. Results for college preparatory students indicate 
that students who successfully completed SLS had a significantly higher Fall- 
to-Winter return rate (87.8% returned) than students who did not take SLS 
(67.6% returned). These results were true across five of the seven 
combinations of deficiencies. For the two exceptions, the sample size was too 
small to test significance for students who were below the placement score on 
writing alone, and a nonsignificant result was obtained for students who 
were below the placement score on reading alone. 

A further examination of success in college preparatory courses indicates that 
students who successfully completed SLS had a significantly higher course 
passing rate (75.6% passed) than students who did not take SLS (62.5% 
passed). These results were true across seven of the nine courses examined. 
The two exceptions were MAT0024 and REA0001 where no significant 
difference was found. 

Finally, results for students on the Standards of Academic Progress indicate 
that those who successfully completed SLS had a significantly higher Faii-to- 
Winter return rate (76.1% returned) than students who did not take SLS (55.8% 
returned). Detailed results are presented below. 

In May of 1998, District Student Services met with Institutional Research to ask for 
an evaluation of the new SLS intervention courses for students who tested into 
college preparatory, and for students on the Standards of Academic Progress 
(SOAP). Three research questions emerged: (1) Did the SLS courses help in the 
retention of college preparatory students? (2) Did the SLS courses help students 
successfully complete college preparatory courses? and (3) Did the SLS courses 
help in the retention of SOAP students? 



Files were accessed using SAS programs specifically written for this purpose The 
college preparatory population consisted of Fall term 1997 first-time-in-college (new) 
students who tested below the placement score on one or more subtests of the CPT. 
Only students with a complete set of scores were included in the study. Further, any 
student who had ESL or ENS courses was eliminated. Of the 7,262 Fall term new 
students, 4,101 were included in the college preparatory study. 
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CPT Status 


Number of 
Students 




Below on one or more subtests 


4,101 


(Study Group) 


Passed all subtests 


614 




No CPT Scores 


1,053 




Partial CPT Scores 


65 




ESL/ENS Courses 


1,429 




Total New Students 


7,262 





The SOAP study consisted of all Opening Fall Term 1997 students who began the 
term in a SOAP category other than Clear. Of the 45,746 credit students enrolled 
Opening Fall 1997, 4,999 were included in the study. 



SOAP Status 
(Opening Fall 97-1) 


Number of 
Students 




Warning 


1,373 


(Study Group) 


Probation 


3,491 


(Study Group) 


Suspension 


135 


(Study Group) 


Clear 


40,747 




Total Opening Fall Students 


45,746 





Results- 
Retention of 
College Preparatory 
Students 



Results- 
Retention of 
SOAP Students 



Table 1 shows the Fall-to-Winter return rate of students who tested into college 
preparatory. Students are separated into groups based on their SLS status. The 
return rate of the group that took and passed SLS is compared with the return rate of 
the group that did not take SLS. The table includes a summary, and a separate 
analysis for each possible combination of CPT subtests failed. The normal 
approximation to the binomial distribution was used to assess the significance of the 
difference between proportions, with the level of significance set at 0.05. 

Note from the summary that only 27% of the 4,101 students below on one or more 
subtest did not take one of the required SLS courses. The Winter return rate of this 
group was 67.6%, significantly below the 87.8% rate for those who took and passed 
SLS1505 orSLS1535. 

The remainder of the Table gives analyses by specific subtests failed. Students 
below on only one area should have taken SLS1505, however some took SLS1535. 
Conversely, students below on two or more areas should have taken SLS1535, but 
some took SLS1505. An additional analysis was performed for the "alternate” 
course when sample size permitted. For five of the seven possible combinations of 
subtests failed, students who took and passed the appropriate SLS course showed a 
significantly higher return rate than those who did not take SLS. In all but one of the 
five combinations (Reading/Writing), the return rate was at least 20 percentage 
points higher. The SLS group showed no significant difference for students who 
failed only the Reading subtest, and the sample size was too small to test results for 
students who failed only the Writing subtest. 

Table 3 shows the Winter return rate of students who began the Fall term in a non- 
Clear SOAP category. The return rate of the group that took and passed SLS1 125 is 
compared with the return rate of the group that did not. 

Note that the proportion of SOAP students who avoided the SLS course (73.4%) is 
much larger than the college preparatory groups analyzed previously. However, the 
summary data and individual category analyses show more than a 20 percentage 
point advantage in return rate for the SOAP students who took SLS. All of the 
comparisons are statistically significant. 
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Conclusions 



A sub-analysis was also done separately for students on SOAP due to withdrawals 
versus GPA. In all cases the return rate of students who took SLS1125 was 
significantly higher than that of students who did not. 

The data presented in this capsule indicate that students who took and passed the 
SLS intervention courses during Fall Term 1997 had a higher Winter Term return 
rate, and were more likely to pass their college preparatory courses than students 
who did not take SLS. The few detailed areas that did not yield significant differences 
will be the topic of discussion by SLS coordinators for possible improvement. 
Additionally, District Student Services is investigating the high proportion of SOAP 
students who did not take the SLS intervention course. 
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Table 1 

Fall-to-Winter Return Rate of First-Time-in-College Students 

By SLS Status 



CPT Test Results Summary: Below on One or More 



SLS Status, Fall 1997-1 


Returned 
Winter 1997-2 




Passed* SLS1505 
No SLS 


564 

1,094 


13.8% 

26.7% 


495 

740 


87.8% 

67.6% 


Significant Difference 
z= 8.905 


Failed** SLS1505 


128 


3.1% 


57 


44.5% 


p-value= 0.0000 


Passed* SLS 1535 1,789 

Failed** SLS1 535 526 

Total 4,101 
‘Grade of 'C or better 
"Any grade other than 'A', 'B', 'C 


43.6% 

12.8% 

100.0% 


1,571 

233 

3,096 


87.8% \Significant Difference 
44.3% z= 13.180 

75.5% p-value= 0.0000 



CPT Test Results: Failed Math only 



SLS Status. Fall 1997-1 


Returned 
Winter 1997-2 




Passed* SLS 1505 


272 


37.1% 


236 


86.8% 


Significant Difference 


No SLS 


319 


43.5% 


211 


66.1% 


z= 5.820 


Failed** SLS1505 


84 


11.5% 


36 


42.9% 


p-value= 0.0000 


Passed* SLS1535 


47 


6.4% 


41 


87.2% \significant Difference 


Failed** SLS1535 


11 


1.5% 


6 


54.5% 


z= 2.915 


Total 


733 


100.0% 


530 


72.3% 


p-value= 0.0018 


•Grade of 'C or better 












“Any grade other than 'A', ’B', 


,'C 










CPT Test Results: 


Failed Reading only 












Returned 




SLS Status. Fall 1997-1 


Winter 1997-2 




Passed* SLS 1505 


155 


47.3% 


140 


90.3% 1 


A/o Significant Difference 


No SLS 


91 


27.7% 


76 


83.5% 1 


z= 1.575 


Failed** SLS1505 


23 


7.0% 


12 


52.2% 


p-value= 0.0576 


Passed* SLS1535 


51 


15.5% 


48 


94.1% 1 


I sample size too small 


Failed** SLS1535 


8 


2.4% 


7 


87.5% 


for normal approximation 


Total 


328 


100.0% 


283 


86.3% 


to the binomial distributioi 



•Grade of 'C or better 

“Any grade other than ’A\ 'B'. 'C 
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Table 1 (Continued) 

Fall-to-Winter Return Rate of First-Time-in-College Students 

By SLS Status 



CPT Test Results: Failed Writing only 





Returned 


SZ.S Status, Fall 1997-1 


Winter 1997-2 



Passed* SLS 1505 


44 


47.3% 


42 


95.5% 


sample size too smalt 


No SLS 


33 


35.5% 


27 


81.8% 


for normal approximation 


Failed** SLS1505 


3 


3.2% 


1 


33.3% 


to the binomial distribution 


Passed* SLS 1535 


11 


11.8% 


9 


81.8% 




Failed** SLS 1535 


2 


2.2% 


1 


50.0% 




Total 


93 


100.0% 


80 


86.0% 




•Grade of *C’ or better 












••Any grade other than 'A', 


•B', 'C 











CPT Test Results; Failed Math/Reading 






SLS Status, Fall 1997-1 


Returned 
Winter 1997-2 




Passed* SLS 1535 


313 55.3% 


282 


90.1% 


Significant Difference 


No SLS 


144 25.4% 


102 


70.8% 


z= 5.222 


Failed** SLS 1535 


69 12.2% 


32 


46.4% 


p-value= 0.0000 


Passed* SLS 1505 


33 5.8% 


27 


8 1 . 8% jyVo Significant Difference 


Failed** SLS 1505 


7 1.2% 


2 


28.6% 


z= 1.280 


Total 


566 100.0% 


445 


78.6% 


p-value= 0.1002 


'Grade of 'C or better 
"Any grade other than 'A', 'B', 


•c 









CPT Test Results: Failed Math/Writing 






SLS Status, Fall 1997-1 


Returned 
Winter 1997-2 




Passed* SLS 1535 


141 


56.2% 


116 


82.3% 


Significant Difference 


No SLS 


58 


23.1% 


36 


62.1% 


z= 3.049 


Failed** SLS1535 


36 


14.3% 


14 


38.9% 


p-value= 0.0110 


Passed* SLS 1505 


13 


5.2% 


13 


100.0% I sample size too small 


Failed** SLS1505 


3 


1.2% 


2 


66.7% 


for normal approximation 


Total 


251 


100.0% 


181 


72.1% 


to the binomial distribution 


'Grade of 'C or better 
••Any grade other than *A’, 
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Table 1 (Continued) 

Fall-to-Winter Return Rate of First-Time-in-College Students 

By SLS Status 



CPT Test Results: Failed Reading/Writing 

Returned 



SLS Status . Fall 1 997-1 Winter 1 997-2 



Passed* SLS1535 


257 


60.9% 


239 


93.0% 


Significant Difference 


No SLS 


91 


21.6% 


73 


80.2% 


z= 3.439 


Failed** SLS1535 


51 


12.1% 


26 


51.0% 


• p-value= 0.0003 


Passed* SLS1505 


21 


5.0% 


19 


90.5% 1 sample size too small 


Failed** SLS1505 


2 


0.5% 


2 


100.0% 


for normal approximation 


Total 


422 


100.0% 


359 


85.1% 


to the binomial distribution. 


‘Grade of 'C or better 












**Any grade other than A', 'B', 'C 










CPT Test Results: Failed All Three 












Returned 




SLS Status , 


Fall 1997-1 


Winter 1997-2 




Passed* SLS1535 


969 


56.7% 


836 


86.3% 


significant Difference 


No SLS 


358 


21.0% 


215 


60.1% 


z= 10.445 


Failed** SLS1535 


349 


20.4% 


147 


42.1% 


p-value= 0.0000 


Passed* SLS1505 


26 


1.5% 


18 


69.2% \No Significant Difference 


Failed** SLS1505 


6 


0.4% 


2 


33.3% 


z= 0.925 


Total 


1,708 


100.0% 


1,218 


71.3% 


p-value= 0.1775 



*Grade of 'C or better 

**Any grade other than 'A', 'B', 'C 
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Table 2 

College Prep Success Rate of First-Time-in-College Students 

By SLS Status 



Summary: All College Preparatory Courses 



SZ.S Status* 


Took Course 


Passed Course ('S') 


1 


Took & Passed 


3,746 61.2% 


2,832 


75.6% 


Significant Difference 


Did Not Take 


1,343 21.9% 


840 


62.5% 


z= 9.157 


Took & Failed 


1,033 16.9% 


279 


27.0% 


p-value= 0.0000 


Total 


6,122 100.0% 


3,951 


64.5% 


'(SLS1505 and SLS1535 combined) 










ENC0002 






SZ.S Status* 


Took Course 


Passed Course ('S') 




Took & Passed 


256 60.2% 


208 


81.3% 


Significant Difference 


Did Not Take 


87 20.5% 


60 


69.0% 


z= 2.395 


Took & Failed 


82 19.3% 


23 


28.0% 


p-value= 0.0083 


Total 


425 100.0% 


291 


68.5% 


'(SLS1505 and SLS1535 combined; 










ENC0020 






SZ.S Status* 


Took Course 


Passed Course ('S') 




Took & Passed 


538 63.8% 


441 


82.0% 


Significant Difference 


Did Not Take 


181 21.5% 


129 


71.3% 


z= 3.072 


Took & Failed 


124 14.7% 


41 


33.1% 


p-value= 0.0011 


Total 


843 100.0% 


611 


72.5% 


*(SLS1505 and SLS1535 combined^ 










ENC0021 






SZ.S Status* 


Took Course 


Passed Course ('S') 




Took & Passed 


296 62.4% 


221 


74.7% 


Significant Difference 


Did Not Take 


83 17.5% 


53 


63.9% 


z= 1.944 


Took & Failed 


95 20.0% 


28 


29.5% 


p-value= 0.0259 


Total 


474 100.0% 


302 


63.7% 


'(SLS1505 and SLS1535 combined; 










MAT0003 






SZ.S Status* 


Took Course 


Passed Course ('S') 




Took & Passed 


301 58.0% 


191 


63.5% 


Significant Difference 


Did Not Take 


124 23.9% 


58 


46.8% 


z= 3.174 


Took & Failed 


94 18.1% 


19 


20.2% 


p-value= 0.0008 


Total 


519 100.0% 


268 


51.6% 


'(SLS1505 and SLS1535 combined; 










o 
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Table 2 (Continued) 

College Prep Success Rate of First-Time-in-College Students 

By SLS Status 

MAT0012 



SLS Status* Took Course Passed Course (’S') 



Took & Passed 359 56.9% 

Did Not Take 151 23.9% 

Took & Failed 121 19.2% 

Total 631 100.0% 

•(SLS1505 and SLS1535 combined; 

MAI 


207 

72 


57.7% 

47.7% 


Significant Difference 
z= 2.067 


20 

299 

r0024 


16.5% 

47.4% 


p-value= 0.0194 


SLS Status* 


Took Course 


Passed Course (’S') 


Took & Passed 


416 


55.0% 


279 


67.1% 1 


\no Significant Difference 


Did Not Take 


243 


32.1% 


151 


62.1% 


\ z= 1.282 


Took & Failed 


98 


12.9% 


24 


24.5% 


p-value= 0.1000 


Total 


757 


100.0% 


454 


60.0% 




•(SLS1505 and SLS 1535 combined; 












REA0001 






SLS Status* 


Took Course 


Passed Course (’S') 




Took & Passed 


407 


65.6% 


324 


79.6% 1 


\no Significant Difference 


Did Not Take 


94 


15.2% 


72 


76.6% 1 


1 z= 0.646 


Took & Failed 


119 


19.2% 


37 


31.1% 


p-value= 0.2590 


Total 


620 


100.0% 


433 


69.8% 




•(SLS1505 and SLS 1535 combined; 












REA0002 






SLS Status* 


Took Course 


Passed Course (’S') 




Took & Passed 


792 


63.1% 


659 


83.2% 


Significant Difference 


Did Not Take 


257 


20.5% 


163 


63.4% 


z= 6.692 


Took & Failed 


206 


16.4% 


61 


29.6% 


p-value= 0.0000 


Total 


1,255 


100.0% 


883 


70.4% 




•(SLS1505 and SLS1535 combined; 












REA0010 






SLS Status* 


Took Course 


Passed Course ('S') 




Took & Passed 


381 


63.7% 


302 


79.3% 


Significant Difference 


Did Not Take 


123 


20.6% 


82 


66.7% 


z= 2.852 


Took & Failed 


94 


15.7% 


26 


27.7% 


p-value= 0.0022 


Total 


598 


100.0% 


410 


68.6% 





•(SLS1505 and SLS1535 combined) 



■r o. 
l\f 
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Table 3 

Fall-to-Winter Return Rate of Students on the Standards of Academic Progress* 

By SLS Status 



Summary of Students on SOAP Opening Fall 



SLS1125 Status, Closing Fall 


Returned 
Winter 1997-2 




Took & Passed 


813 


16.3% 


619 


76.1% 


Significant Difference 


Did Not Take 


3,667 


73.4% 


2,046 


55.8% 


z= 10.690 


Took & Failed 


519 


10.4% 


174 


33.5% 


p-value= 0.0000 


Total 


4,999 


100.0% 


2,839 


56.8% 



SOAP Status Opening Fall: Warning 



SLS1125 Status, Closing Fall 


Returned 
Winter 1997-2 




Took & Passed 


207 


15.1% 


162 


78.3% 


Significant Difference 


Did Not Take 


988 


72.0% 


577 


58.4% 


z= 5.348 


Took & Failed 


178 


13.0% 


64 


36.0% 


p-value= 0.0000 


Total 


1,373 


100.0% 


803 


58.5% 





SOAP Status Opening Fall: Probation 



SLS1125 Status, Closing Fall 


Returned 
Winter 1997-2 




Took & Passed 


574 


16.4% 


436 


76.0% 


Significant Difference 


Did Not T ake 


2,602 


74.5% 


1443 


55.5% 


z= 9.045 


Took & Failed 


315 


9.0% 


107 


34.0% 


p-value= 0.0000 


Total 


3,491 


100.0% 


1986 


56.9% 





SOAP Status Opening Fall: Suspension 



SLS1125 Status , Closing Fall 


Returned 
Winter 1997-2 




Took & Passed 


32 


23.7% 


21 


65.6% 


Significant Difference 


Did Not Take 


77 


57.0% 


26 


33.8% 


z= 3.059 


Took & Failed 


26 


19.3% 


3 


11.5% 


p-value= 0.0220 


Total 


135 


100.0% 


50 


37.0% 
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